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Abstract 



We present a local algorithm for finding dense subgraphs of bipartite 
graphs, according to the definition of density proposed by Kannan and 
Vinay. Our algorithm takes as input a bipartite graph with a specified 
starting vertex, and attempts to find a dense subgraph near that vertex. 
We prove that for any subgraph 5* with k vertices and density 9, there are 
a significant number of starting vertices within S for which our algorithm 
produces a subgraph S" with density Q{6/ logn) on at most O(Afc^) ver- 
tices, where A is the maximum degree. The running time of the algorithm 
is O(Afc^), independent of the number of vertices in the graph. 



1 Introduction 

Identifying dense subgraphs has become an important task in the analysis of 
large networks, and a collection of dense subgraphs may reveal a wealth of 
information about a graph. In particular, dense subgraphs often form the cores 
of larger communities or clusters in the graph [9J . 

Kannan and Vinay [8 introduced a notion of density that is well-suited to 
bipartite graphs representing incidence matrices. As an example, consider a 
bipartite graph describing the incidences between a set of groups G and a set 
of group members M . The density of the subgraph induced by a set of groups 
S C Q and a set of members TQM, is defined to be 



which is the total number of incidences between the groups and members in the 
subgraph, divided by the geometric mean of the number of groups and number of 
members in the subgraph. There are a variety of efficient algorithms for finding 
a subgraph with nearly optimal density according to this definition. Kannan and 
Vinay gave a spectral algorithm that produces from the largest eigenvector of A 



e{S,T) 




a subgraph whose density is within an O(logrt) factor of optimal. Charikar [5] 
showed that a subgraph with optimal density can be identified in polynomial 
time by solving a linear program, and also gave a greedy algorithm that produces 
a 2-approximation of the densest subgraph in linear time. 

In this paper, we present a local algorithm for finding dense subgraphs. 
Our algorithm takes as input a graph with a specified starting vertex, and 
attempts to find a dense subgraph near that vertex. We prove the following 
local approximation guarantee for our algorithm: for any subgraph H with 
density 0, there are a significant number of starting vertices within H for which 
our algorithm produces a subgraph with density ^{6 / logn). The running time 
of the algorithm is O(Afc^), where k is the number of vertices in iJ, and where 
A is the maximum degree in the graph. 

There are two principal tasks that our local algorithm can perform which, 
to our knowledge, can not be accomplished by other known algorithms for the 
densest subgraph problem. The first is to find a dense subgraph near a vertex 
of interest, while examining only a portion of the entire graph. The second is 
to find many small dense subgraphs in parallel, which we can accomplish by 
applying the local algorithm at many different starting vertices. In addition, 
our algorithm provides an upper bound on the size of the subgraph it produces, 
which might make it a useful theoretical tool for producing a dense subgraph of 
a specified size. 

To analyze our algorithm, we build upon the spectral techniques developed 
by Kannan and Vinay, exploiting the close relationship between the densest 
subgraph of a graph and the largest eigenvalue of the graph's adjacency matrix. 
We define a deterministic process called the 'pruned growth process', which 
produces a sequence of vectors, and show that by computing those vectors we 
can identify a subgraph with high density. We show that these vectors can be 
rounded at each step to ensure that the number of nonzero elements is small, 
which decreases the time required to compute them. A similar type of local 
approximation algorithm has been developed for the related problem of graph 
partitioning [TUl [J . The densest subgraph problem is the second problem for 
which this type of local spectral algorithm has been developed. 

In Section [2j we state the definition of density introduced by Kannan and 
Vinay, compare this definition with others that have appeared in the literature, 
and survey known algorithms for the densest subgraph problem. In Section [31 
we define the 'pruned growth process'. In Section^ we state our local algorithm 
and analyze its running time and approximation guarantee. In Section [51 we 
describe an efficient global approximation algorithm for the densest subgraph 
problem, which will follow easily from our work in the previous sections. 
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2 Preliminaries and Related Work 



Let G = {y,E) be an undirected bipartite graph with adjacency matrix A, and 
let L and R be the left and right sides of a fixed bipartition. The edges of the 
graph may be weighted, in which case the entry Aij is the weight of edge {i, j}. 
For any two sets S C L and T C i?, we let {S, T) denote the induced bipartite 
subgraph of G on the set of vertices 5 U T, and we define e(5', T) to be the sum 
of the weights of the edges between S and T. We will sometimes use the inner 
product notation e{S,T) = {Is A, It), where I5 is the indicator function for 
membership in S. We define the support of a vector x to be the set of vertices 
on which x is nonzero. 

We will identify induced subgraphs of G which are dense according the fol- 
lowing definition, which was introduced by Kannan and Vinay [8]. 

Definition 1. For any induced subgraph (S,T), we define 



W\T\ 

We define d(A) to be the maximum value of d{S,T) over all induced subgraphs. 



Our algorithm may also be applied to an arbitrary directed graph, using the 
following trick. Given a directed graph with vertex set X, define a bipartite 
graph where L — R = X . For each edge x y in the directed graph, place an 
undirected edge between the copy of x in L and the copy of y in R. 



2.1 Related work 



A different definition of density was considered in [II El [2] . 

Definition 2. Let G = (V, E) be an undirected graph (not necessarily bipartite). 
For any set S Q V , we define 

= 

We define g{A) to be the maximum value of g{S) over all subsets of V . 



Both d{A) and g{A) can be computed exactly in polynomial time. Goldberg 
showed that a set S achieving g{S) = g{A) can be found using maximum flow 
computations 7 . Such a set can also be found using the parametric flow algo- 
rithm of Gallo, Grigoriadis, and Tarjan [5_. Charikar showed that a subgraph 
(5*, T) achieving d{S, T) = d{A) can be found by solving a linear program [2]. 

Charikar gave greedy 2-approximation algorithms for both g{A) and d{A) 
[2]. The running time of these algorithms is 0{m) in an unweighted graph, and 
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0(m + nlogn) in a weighted graph. Kannan and Vinay gave a spectral ap- 
proximation algorithm for d{A), which produces a subgraph [S, T) with density 
d{S,T) = ^l{d{A)/logn) from the top singular vectors of A. 

The closely related densest fc-subgraph problem is to identify the subgraph 
with the largest number of edges among all subgraphs of exactly k vertices. 
This problem is considerably more difficult, and there is a large gap between 
the best approximation algorithms and hardness results known for the problem 
(see [2 [3]). 



2.2 Comparison oi d{S,T) and g{S) 



It is easier to compare the two objective functions ^(5*, T) and g{S) if we restrict 
g{S) to bipartite graphs. In this case, g{S) takes the following form. 

Definition 3. For any subgraph {S,T), we define 

9i^'-^)- \S\ + \T\- 
We define g{A) to be the maximum value of g{S,T) over all induced subgraphs. 



The two objective functions d(S, T) and g{S, T) are far apart when S and T 
have very different sizes. The quantities d{A) and g{A) can also be far apart. In 
the complete bipartite graph -fCa,fc, we have d{A) — \foh, while ^(A) — ab/ {a + 6). 
In the case where a = 1, we have d{A) = \/b while g{A) = 6/(6 + 1) ^ 1. 

The relative merits of d{S,T) and g{S) as objective functions for density 
were discussed in [31 [5]. In this paper, we consider d{S,T) because it is more 
amenable to approximation by spectral algorithms than g{S), not because we 
prefer it as an objective function. The largest eigenvalue of the adjacency matrix 
A is closely related to d{A). We know of no similar result for g{A), and we do 
not know how to produce a local algorithm for the objective function g{S). 



3 The pruned growth process 



We now define the deterministic process that will be the basis for our local 
algorithm. The process generates a sequence of vectors Xq, . . . ,xt from a start- 
ing vector a;o. The main operation performed at each step is multiplication by 
the adjacency matrix A, as in the power method. The resulting vector is then 
rounded by making each entry a power of 2, and then pruned by setting to 
zero each entry whose value is below a certain threshold. These steps reduce 
the number of possible values in the vector and reduce the size of the support, 
minimizing the amount of computation required. 
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Definition 4. Given a vector z, we define round(z) to be the vector obtained 
by rounding each entry of the vector z up to the nearest power of 2, 

[round(2)](w) = 2*, where i is the smallest integer such that 2* > z{u). 

Given a vector z and a nonnegative real number e, we define prune^ (z) to 
be the vector obtained by setting to zero any entry of z whose value is at most 

z{u) if z{u) > e\\z\\ , 
otherwise. 



[prunee {z)]{u) 



Definition 5. Given a starting vector xq with entries from {0,1}, and a se- 
quence of real numbers et € [0, 1], we define the pruned growth process to be the 
sequence of vectors xq,. . . ,xt defined by the following rule: 

Xt+i = pruneej^j(round(xtA)). 

Notice that each entry of Xt+i is either zero or a power of two. 

Definition 6. Given the vectors xq, ■ ■ ■ ,xt of the pruned growth process, we 
define X* to he the set of vertices where Xt{v) = 2*, and define to be the set 
of vertices where iomid{xtA)[v) = 2\ 

Wc will eventually show that a subgraph with high density can be found 
whenever the norms of the pruned growth process vectors grow quickly. 
The following lemma shows that if none of the subgraphs {X-,Yj) has high 
density, then ||a;t|| is not much larger than ||a;t-i||. 

Lemma 1. If d{Xl,Y^) < 6 for all i,j, then, 

\\xt+i\\ < \\iound{xtA)\\ < 20\\xt\\\og—. 

Proof. We will write || round(a;tA)|p in terms of the densities d{Xl, Yj). 

II round(a;(A)||^ = (round(a;tA),round(a;tA)) 
< {2xtA,xomiA.{xtA)) 



= /2^2'lx*A,^2ny* 

\ j 3 

= 2^2'2^' UxlA, lyt 




5 



In the sum above, we need only sum over those i where Xf is nonempty. There 
are at most log ^ such values, because every nonzero value in xt is at most ||xt|| 
and at least et||a;t||. We now apply the Cauchy-Schwarz inequality to show 



Similarly, we need only sum over those j where is nonempty. There are at 
most log ^ such values, because every nonzero value of round(a;tv4) is at most 
||round(a;tJl)|| < 2A||xt||, and at least et||a;t_i||. We apply the Cauchy-Schwarz 
inequality again to show 

1/2 / X 1/2 



E < I El^*|2''' I I El I ^ llroundMll^log^. 
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Then, 



||round(a;tA)||^ < 2e||a;t|||| round(a;tA)||y'log log ^ 

2A 

< 26»||a;t||||round(a;tA)||log — . 
The lemma follows. □ 



4 Local approximation algorithm 

In this section, we will state and analyze a local algorithm for finding dense 
subgraphs. The input to the algorithm is a graph, along with a starting vertex 
V and a target size K. We will prove that the running time of the algorithm 
depends mainly on the target size K, and is independent of the number of 
vertices in the graph. We will prove that for any subgraph (S*, T), there are a 
significant number of starting vertices in S for which the algorithm produces a 
subgraph whose density is within an O(logn) factor of d{S, T). 

LocalDensity(v, K) 

Input: A vertex v and a target size K. 

Output: A subgraph {X,Y). 

1. Let xo = It,, let T = \og{y^2\K\), and let e* = ^^'K 

2. Compute the vectors xq, . . . ^xt of the pruned growth process. 

3. Compute d{Xl, Yj) for each pair i, j and each time t < T. 

4. Output the subgraph with the highest density. 
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Theorem 1. Let {S,T) be a subgraph such that d{S,T) > 29. Then there exists 
a set Sg C S , with the following properties. 

1. e(Se,T)>e(5,r), 

2. If V Cz Se and K > niaxdS*!, |T|), then LocalDensity(v, K) outputs a sub- 
graph {X, Y) such that 

d(X,Y) > = n(-^). 

^ ' ' - 81ogl6AX nogn^ 

Theorem 2. LocalDensity(v, K) runs in time 0{AK^). 



The proofs of Theorems [T] and [2] are given in section 14.21 



4.1 Lower bounds on growth within a dense subgraph 

The main step in analyzing the algorithm LocalDensity is to prove a lower 
bound on the growth of the norms \\xt\\ . We will use the fact that the maximum 
density d{A) gives a lower bound on the largest eigenvalue of A. 

Fact 1. Let A be the adjacency matrix of an undirected graph, and let A be the 
largest eigenvalue of A. Then, A > d{A). Furthermore, there is an eigenvector 
(j) with eigenvalue A whose entries are nonnegative. 



Proof. To prove that A > d{A), notice that for any sets S C L and T C 
A > -X > (^A, . = diS,T). 

~ \M\\y\\ ~\^M WW\I 

It is not hard to see that if (j) is an eigenvector with eigenvalue A, then the vector 
whose entries are the absolute values of the entries of (f) is also an eigenvector 
with eigenvalue A. □ 



The fact above implies the lower bound ||a;oj4*|| > {(f) , xq) d{Ay , which de- 
pends on the maximum density d{A). To analyze the local algorithm, we will 
give a lower bound that depends on the density of a particular subgraph {S, T) 
containing the starting vertex. Specifically, we will show that for many vertices 
in the set S', we can give a bound of the form ||xoA*|| = {l{d{S, T)*) with a not- 
too-small constant term. We will do so by considering how the pruned growth 
process would behave if it were restricted to the induced subgraph {S,T). 

Definition 7. For any induced subgraph (S,T), we define ^(5,t) to be the 
restriction of the adjacency matrix A to {S,T), 



A{x, y) if X £ S and y £T , or if x £T and y E S. 
otherwise. 
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The following lemma identifies, for any subgraph {S,T), a set of starting 
vertices for which we can give a good lower bound on the norms |ia;t||. This 
set of good starting vertices touches at least half of the edges in the induced 
subgraph {S,T). 

Lemma 2. // {S,T) is a subgraph such that d{S,T) > 29, then there exists a 
subset Se C S* with the following properties: 



1. e(Se,T)>e(5,r)/2 

2. For each t; G Sg, there is a nonnegative unit vector ip such that 

(a) Support('0) CSUT, 

(b) il;A>9i^, 

(c) i;iv) > 



/2\S\ 



Proof. Let Sg be the largest subset of S for which property (2) holds, and 
consider the set S' = S \ Sg. If does not satisfy property (1), then 

e(5',T) = e(5,T)-e(S,,r)>^^^, 

and so d{S',T) > d{S,T)/2 > 0. 

Let A be the largest eigenvalue of v4(s'.t)- We know from fact [T] that there 
is an eigenvector ip of A(^si t) whose entries are all nonnegative, and whose 
corresponding eigenvalue A satisfies 

A > d{S',T) > 9. 

It is easy to see that ip satisfies properties (a) and (b). We will now identify a 
vertex in S' for which ^{v) > 1/ ■\/2|S'|. This will imply that v is in Sg, which 
will show that Sg must satisfy property (1), and thus complete the proof. 

Let and ipx be the projections of ip onto S" and T, and observe that 
llV'S'll — IIV'tII — This is true because '<Ps'^{S',T) = M^T: which implies that 
AllV'S'll — II^S'^(S',T)|| = -^IIVtII- There must at least one vertex v in 5" which 
satisfies > l/^/2\S'\, since otherwise we would have ||i/'S'|P < 1/2. □ 



4.2 Analysis of the local algorithm 

Proof of Theorem [H We will prove that for each vertex v in the set Sg, which 
was described in Lemma [21 the algorithm LocalDensity(v, K) outputs a sub- 
graph with density at least 6'/8L, where L = log(2A/eo) < (log IGAX), provided 
that K > maxdS"!, |T|). The theorem will follow. 
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Let xo,. . . ,xt be the pruned growth process vectors computed by the algo- 
rithm. We will assume that the algorithm does not find a subgraph with the 
desired density, and derive a contradiction. That is, we assume that for each 
i, j, and each time t < T, we have d{Xl,Yj) < Under this assumption. 
Lemma [1] shows that for every t <T, 

\\xt+i\\ < II round{xtA)\\ 

<(..o.f)(A)„.„ 

Since ||a;o|| ~ 1, this implies 



|a;f II < II round(xt_iA)|| < ( ^ ) for every t <T. (1) 



Since v G Sg, there exists a nonnegative vector ip such that ipA > dip, such 
that Support('0) C S* U T, and such that ip{v) > "yjj^i stated in Lemma [2j 

We will prove the following lower bound on the inner product of Xt with ip. 

{xt^ip) > i (6>/2)* for every t < T. (2) 



When we prove equation it will contradict equation ([T]) when t — T = 
\og{^j2\S\), and we will be done. 

We will prove that equation ([2]) holds by induction. We know it holds for 
t = Q. The only difficulty in the induction step is to bound the effect of the 
pruning step on the projection of xt onto ip. We define rt to be the vector that 
is removed during the pruning step. 

rt = round(a;f_iA) - Xt 

~ round(xt_iA) — prunc^j (round(a;t_iA)). 



The value of rt at any given vertex is at most et||round(a;t_iyl.)|| . Since the 
support of -0 is contained in 5 U T, and the support of rt is contained in either 
L or i?, the intersection of the two supports contains at most maxd^l, |T|) 
vertices. The inner product of rt and ■0 can then be bounded as follows. 

{rt,'P) < et||round(a;f_iA)|| v/Support(rf) n Support('0) 
< et\\TOund{xt-iA)\\y/K. 
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We can now bound {xt, ijj) in terms of {xt-i, 4')- 

{xt,i>) = (TOuiid{xt-iA) -rt,i>) 

= (round(a;4_iyl),i/') - {rt,tp) 

> 9 (xt^i^Tp) ~ et||round(a;t-iA|| )Vk. 

We now assume that the induction hypothesis holds for t — 1, which means 
{xt-iT^j) ^ (1/a/2|S'|)(0/2)*~-'-. Recall that we have assumed for the sake of 
contradiction that < || round(a:t_iA)|| < (0/4)*. We will now show that 
the induction hypothesis holds for t. 

<->^fe(^)'1-HG)') 

The last step follows because we have set cj so that 

_ 2-* 

This completes the proof. □ 

Proof of Theorem [2l We bound the running time of LocalDensity(v, K) by 

bounding the number of vertices in the support of xt at each step. Since xt is 
at least e||xt|| wherever it is nonzero, we have 

\\xt\\^ > |Support(a;t)|e'||a:t||2, 

and so ^ 

I Support (xt) I < ^. 

We can compute xt+i from xt and compute the density of each subgraph 
{Xf,Yj) in time proportional to the sum of the degrees of the vertices in 
Support (xt), which is at most 

0(A|Support(xt)|) = 0(A/e2) = 0{AK^2~^*). 

The total running time is therefore 

T 

t=o 

□ 
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5 An approximation algorithm for d{A) 



As a simple application of the techniques developed in the previous sections, 
we give an 0(log n)-approximation algorithm for the globally optimum density 
d{A) by simulating the pruned growth process for O(logn) steps. The algorithm 
produces a subgraph {S,T) with density il,{d{A)/ \ogn) in time 0{m\ogA/d), 
where A is the maximum degree in the graph, and d is the average degree. 
The algorithm requires O(logn) passes through the collection of adjacency lists 
describing the graph, and requires only 0(n log log n) bits of additional storage. 
This provides an efficient way to implement the spectral approximation algo- 
rithm of Kannan and Vinay [8], which has the same O(logn) approximation 
guarantee and requires computing the largest eigenvalue of A. 



Density: 

Run the following procedure twice with xq — II and xq — In: 

1. Let T = log2V^ and 

2. Compute the pruned growth process vectors Xq, . . . ,Xt- 

3. Compute d{X^, Yj) for each pair i, j and each time t < T. 

4. Output the densest subgraph among the sets (X*,y^*). 



Theorem 3. For at least one of the two starting vectors 1l and Ir, there exists 
a time t < T and two indices i and j such that the subgraph {X, Y) output by 
the algorithm satisfies 

A d(A) 



(8 + 41ogn) ~ (8 + 41ogn)' 

Theorem 4. Density runs in time 0{m{l + log^)), where A is the maxi- 
mum degree in the graph, and d is the average degree. The algorithm requires 
0(n log log n) bits of additional storage. 



Proof of Theorem \3[ Let A be the largest eigenvalue of A, and let 4> be an 
eigenvector with eigenvalue A whose entries are nonnegative. Because (fi is non- 
negative, > 1. We will assume that 1l has a larger inner product with 
(j) than so that > (1/2) (lv,(/') > 1/2. We let xq = 1l, and consider 
the vectors xq, . . . ,xt computed by the algorithm. 

We assume that y/) < A/8 log(2A/eo) < A/(8 + 41ogn) for every i, j, 

and t <T, and derive a contradiction. Under this assumption, Lemma [1] shows 
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that for every t <T, 



\\xt+i\\ < \\T:onnd{xtA)\ 
2AW 

et ) V8fog(2A/eo) 
^ 1 l) ll-l 



Since ||a;o|| < \/n, this imphes 

ll-'ctll < for every i<T. (3) 

We will soon prove the following lower bound. 

{xu 4>) > 0) (A/2)* for every t < T. (4) 
When t = T = log(2Vn), this will imply 

\\xt\\ >(2:T,0)>^(A/2f >y^(A/4f, 

which will contradict equation ([3]), completing the proof. 

We will prove by induction that equation ^ holds for every t <T. It holds 
trivially for t = 0. We define rt to be the vector lost in the pruning step, 

rt = round(a;t_iA) - Xt 

= round(a;(_i A) — prune^j (round(xt_i A)). 

The value of rt at any given vertex is at most et||round(a;t_iA)|| < 2Aef ||a;t_i|| . 
Because is nonnegative, {rt,4') < 2Ae4||xf_i|| In fact, we have the 

slightly stronger statement {rt,(f)) < 2Aei||xt_i|| (1^,0), because the support of 
rt is contained in either L or R, and 1^ has a larger inner product with cf). We 
can now bound {xt, 4>) in terms of {xt-i, (f)) . 

{xt, (f)) = (round(a;t_i^) -rt,cj)) 

= (round(a;f_iA), (/)) - {rt,(j)) 
>X{xt^i,(j>) -2Xet\\xt-i\\ (11,^)- 

We will assume that the induction hypothesis holds for t — 1, which means that 
{xt-i,(j>) > {^LT<t>) (A/2)*"-'^, and we have already assumed for the sake of con- 
tradiction that ||xf|| < \/n(A/4)*. We now show that the induction hypothesis 
holds for t. 

{xt,4>) > A 0) (A/2)*-i - 2Aet (U, 0) V^(A/4)*-i 

> (xo,^) (A/2)*(2-8et2-*V^) 

> (xo,^) (A/2)*. 
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The last step follows because we have set so that 

2* 



This completes the proof. □ 

Proof of Theorem [4l We can bound the runmng time of the algorithm by 
bounding the number of vertices in the support of xt- Since xt is at least et||a;(|| 
wherever it is nonzero, we have 

\\xt\\'>\Support{xt)\e^,\\xtf, 

and so 



|Support(xt)| < 4 < "2-2(*-3) 



We can compute Xt+i from xt and compute the density of each subgraph 
{Xf, Yj) in time proportional to the number of edges incident with Support (a;t), 

|e(Support(xt), y)| < min(m, A|Support(a;t)|) 
< min(m, An2"^(*~^)). 

The total running time over all T steps is at most 

^ /I \ ^ 

min(m, An2-^^'-^^) < i - log(nA/m) + 3 j m + ^ An2-^^*'^'> 

*=0 ^ ^ t=i log(nA/m)+3 

< ^- log(nA/m) + 3^ m + 2m 
= 0(mlog(A/fi) + m). 

To bound the amount of space used by the algorithm, notice that storing the 
vector Xt requires n log log i = O(nloglogn) bits, since each vertex takes one 
of log i possible values. We need only store two vectors at a given time, Xt and 
round(a;fA), so the total amount of storage required is O(nloglogn) bits. □ 



6 Conclusion 



We have shown that it is possible to find a dense subgraph near a given vertex 
without examining the entire graph. The running time of our local algorithm 
is quadratic in terms of the target size K, where K must be at least as large as 
\S\ + \T\ to produce an approximation of the subgraph {S,T). We conjecture 
that a better local algorithm exists. In particular, it would be nice to have an 
algorithm whose running time depends on the size of the subgraph {X, Y) that 
is produced, rather the subgraph {S, T) whose density is approximated. 
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